IEEE Transactions on Computational Biology and Bioinformatics

Hypergraphs are powerful tools for modeling complex interactions across various domains, including biomedicine. However, learning meaningful node representations from hypergraphs remains a challenge. Existing supervised methods often lack generalizability, thereby limiting their real-world applications. We propose a new method, Pre-trained Hypergraph Convolutional Neural Networks with Self-supervised Learning (PhyGCN), which leverages hypergraph structure for self-supervision to enhance node representations. PhyGCN introduces a unique training strategy that integrates variable hyperedge sizes with self-supervised learning, enabling improved generalization to unseen data. Applications on multi-way chromatin interactions and polypharmacy side-effects demonstrate the effectiveness of PhyGCN. As a generic framework for high-order interaction datasets with abundant unlabeled data, PhyGCN holds strong potential for enhancing hypergraph node representations across various domains.

Synthetic lethality (SL) offers a promising approach for targeted cancer therapies. Current SL prediction models heavily rely on extensive labeled data for specific cell lines to accurately identify SL pairs. However, a major limitation is the scarcity of SL labels across most cell lines, which makes it challenging to predict SL pairs for target cell lines with limited or even no available labels in real-world scenarios. Furthermore, gene interactions could be opposite between training and test cell lines, i.e. SL vs. non-SL, which further aggravates the challenge of generalization among cell lines. A promising strategy is to transfer knowledge learned from cell lines with relatively abundant SL labels to those with limited SL labels for the discovery of novel SL pairs, i.e., cell line-adapted SL prediction. Here, we propose MiT4SL, a multi-omics triplet representation learning model for cell line-adapted SL prediction. The core idea of MiT4SL is to model cell lineage information as embeddings, which are generated by combining a protein-protein interaction network representation tailored to each cell line with the corresponding protein sequence embeddings. We then combine these cell line embeddings with gene pair representations derived from a biomedical knowledge graph and protein sequences. This triplet representation learning strategy enables MiT4SL to capture both shared biological mechanisms across cell lines and those unique to each cell line, effectively mitigating distribution shift and improving generalization to target cell lines. Additionally, explicit cell line embeddings provide the necessary signals for MiT4SL to effectively differentiate between cell line contexts, enabling it to adjust predictions and mitigate possible label conflicts for the same gene pair across different cell lines. Experimental results across various cell line-adapted scenarios show that MiT4SL outperforms six state-of-the-art models. To the best of our knowledge, MiT4SL is the first deep learning model designed specifically for cancer cell line-adapted SL prediction. AvailabilityThe code of our work is available at https://github.com/JieZheng-ShanghaiTech/MiT4SL.

Enhancers are important cis-regulatory elements, enhancing the transcription of target genes. De novo design of high-activity enhancers is one of long-standing goals in generated biology for both clinical purpose and artificial life, because of their vital roles on regulation of cell development, differentiation, and apoptosis. But designing the enhancers with specific properties remains challenging, primarily due to the unclear understanding of enhancer regulatory codes. Here, we propose an AI-driven enhancer design method, named Enhancer-GAN, to generate high-activity enhancer sequences. Enhancer-GAN is firstly pre-trained on a large enhancer dataset that contains both low-activity and high-activity enhancers, and then is optimized to generate high-activity enhancers with feedback-loop mechanism. Domain constraint and curriculum learning were introduced into Enhancer-GAN to alleviate the noise from feedback loop and accelerate the training convergence. Experimental results on benchmark datasets demonstrate that the activity of generated enhancers is significantly higher than ones in benchmark dataset. Besides, we find 10 new motifs from generated high-activity enhancers. These results demonstrate Enhancer-GAN is promising to generate and optimize bio-sequences with desired properties.

Learning low-dimensional continuous vector representation for short k-mers divided from long DNA sequences is key to DNA sequence modeling that can be utilized in many bioinformatics investigations, such as DNA sequence classification and retrieval. DNA2Vec is the most widely used method for DNA sequence embedding. However, it poorly scales to large data sets due to its extremely long training time in kmer embedding. In this paper, we propose a novel efficient graph-based kmer embedding method, named Kmer-Node2Vec, to tackle this concern. Our method converts the large DNA corpus into one kmer co-occurrence graph and extracts kmer relation on the graph by random walks to learn fast and high-quality kmer embedding. Extensive experiments show that our method is faster than DNA2Vec by 29 times for training on a 4GB data set, and on par with DNA2Vec in terms of task-specific accuracy of sequence retrieval and classification.

Learning generalizable, transferable, and robust representations for molecule data has always been a challenge. The recent success of contrastive learning (CL) for self-supervised graph representation learning provides a novel perspective to learn molecule representations. The most prevailing graph CL framework is to maximize the agreement of representations in different augmented graph views. However, existing graph CL frameworks usually adopt stochastic augmentations or schemes according to pre-defined rules on the input graph to obtain different graph views in various scales (e.g. node, edge, and subgraph), which may destroy topological semantemes and domain prior in molecule data, leading to suboptimal performance. Therefore, designing parameterized, learnable, and explainable augmentation is quite necessary for molecular graph contrastive learning. A well-designed parameterized augmentation scheme can preserve chemically meaningful structural information and intrinsically essential attributes for molecule graphs, which helps to learn representations that are insensitive to perturbation on unimportant atoms and bonds. In this paper, we propose a novel Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations, MolCLE for brevity, that self-adaptively incorporates chemically significative information from both topological and semantic aspects of molecular graphs. Specifically, we apply deep neural networks to parameterize the augmentation process for both the molecular graph topology and atom attributes, to highlight contributive molecular substructures and recognize underlying chemical semantemes. Comprehensive experiments on a variety of real-world datasets demonstrate that our proposed method consistently outperforms compared baselines, which verifies the effectiveness of the proposed framework. Detailedly, our self-supervised MolCLE model surpasses many supervised counterparts, and meanwhile only uses hundreds of thousands of parameters to achieve comparative results against the state-of-the-art baseline, which has tens of millions of parameters. We also provide detailed case studies to validate the explainability of augmented graph views. CCS CONCEPTS* Mathematics of computing [->] Graph algorithms; * Applied computing [->] Bioinformatics; * Computing methodologies [->] Neural networks; Unsupervised learning.

Protein functions can be described by the Gene Ontology (GO) terms, allowing us to compare the functions of two proteins by measuring the similarity of the terms assigned to them. Recent works have applied neural network models to derive the vector representations for GO terms and compute similarity scores for these terms by comparing their vector embeddings. There are two typical ways to embed GO terms into vectors; a model can either embed the definitions of the terms or the topology of the terms in the ontology. In this paper, we design three tasks to critically evaluate the GO embeddings of two recent neural network models, and further introduce additional models for embedding GO terms, adapted from three popular neural network frameworks: Graph Convolution Network (GCN), Embeddings from Language Models (ELMo), and Bidirectional Encoder Representations from Transformers (BERT), which have not yet been explored in previous works. Task 1 studies edge cases where the GO embeddings may not provide meaningful similarity scores for GO terms. We find that all neural network based methods fail to produce high similarity scores for related terms when these terms have low Information Content values. Task 2 is a canonical task which estimates how well GO embeddings can compare functions of two orthologous genes or two interacting proteins. The best neural network methods for this task are those that embed GO terms using their definitions, and the differences among such methods are small. Task 3 evaluates how GO embeddings affect the performance of GO annotation methods, which predict whether a protein should be labeled by certain GO terms. When the annotation datasets contain many samples for each GO label, GO embeddings do not improve the classification accuracy. Machine learning GO annotation methods often remove rare GO labels from the training datasets so that the model parameters can be efficiently trained. We evaluate whether GO embeddings can improve prediction of rare labels unseen in the training datasets, and find that GO embeddings based on the BERT framework achieve the best results in this setting. We present our embedding methods and three evaluation tasks as the basis for future research on this topic.

Pre-trained large language models demonstrate potential in extracting information from DNA sequences, yet adapting to a variety of tasks and data modalities remains a challenge. To address this, we propose DNAGPT, a generalized DNA pre-training model trained on over 200 billion base pairs from all mammals. By enhancing the classic GPT model with a binary classification task (DNA sequence order), a numerical regression task (guanine-cytosine content prediction), and a comprehensive token language, DNAGPT can handle versatile DNA analysis tasks while processing both sequence and numerical data. Our evaluation of genomic signal and region recognition, mRNA abundance regression, and artificial genome generation tasks demonstrates DNAGPTs superior performance compared to existing models designed for specific downstream tasks, benefiting from pre-training using the newly designed model structure.

In biomedical research, the utilization of Knowledge Graph (KG) has proven valuable in gaining deep understanding of various processes. In this study, we constructed a comprehensive biomedical KG, named as MegaKG, by integrating a total of 23 primary data sources, which finally consisted of 188, 844 nodes/entities and 9, 165, 855 edges/relations after stringent data processing. Such a massive KG can not only provide a holistic view of the entities of interest, but also generate insightful hypotheses on unknown relations by applying AI computations. We focused on the interplay of the key elements in drug development, such as genes, diseases and drugs, and aimed to facilitate practical applications that could benefit early drug development in industries. More importantly, we placed much emphasis on the exploitability of the predictions generated by MegaKG. This may greatly help researchers to assess the feasibility or design appropriate downstream validation experiments, making AI techniques more than just black-box models. In this regard, NBFNet was adopted, which combines the advantages of both traditional path-based methods and more recently developed GNN-based ones. Performance evaluation experiments indicated superior results by MegaKG. We also conducted real case studies to validate its practical utility in various scenarios, including target prediction, indication extension and drug repurposing. All these experiments highlighted the potential of MegaKG as a valuable tool in driving innovation and accelerating drug development in pharmaceutical industry.

Clinical studies have shown that microbes are closely related to the occurrence of diseases in the human body. It is beneficial for treating diseases by means of microbes to modulate the activity and toxicity of drugs. Therefore, it is significant in predicting associations between drugs and microbes. Recently, there are several computational models for addressing the issue. However, most of them only focus on drug-related microbes and neglect related diseases, which can lead to insufficient training. Here we introduce a new model (called MDMD) is proposed to predict drug-related microbes based on the Metapaths from a heterogeneous network constructed by using the data of Diseases, Microbes, Drugs, the associations of microbe-disease and disease-drug. The MDMD uses an aggregation of the metapath features that can effectively abundance the embedding of the features for different types of nodes and edges in the heterogeneous networks. Then, the MDMD uses the attention mechanism to mark the importance of the metapath vector for each node type which can improve the quality of feature embedding. Experimental results demonstrate that the MDMD improves accuracy by 1.9% compared with other models. The MDMD is also used to predict the microbes of two drugs Lamivudine and Tenofovir which are the antiretroviral drugs used to treat the Acquired Immune Deficiency Syndrome(AIDS). The results show that 90-95% of microbes are reported in the PubMed. Mycobacterium tuberculosis(Mtb) is a specific microbe only predicted by the MDMD. An online platform of the MDMD is available in https://mdmd2023.bit1024.top/, in which the source code of the MDMD and the data in the work can be downloaded. Author summaryMicrobes inhabit multiple organs of the human body that consist of bacteria, fungi, and viruses. Extensive research shows that the microbes can adjust the efficacy and toxicity of drugs to treat the disease. The efficient and accurate selection of drug-related microbes is important for drug research and disease treatment. However, screening of drug-related microbes relies on traditional lab experiments that are labor-intensive and costly. With the growth of high-throughput data, the research of drug-related microbes urgently needs a computational method in bioinformatics. However, most of them only focus on drug-related microbes and neglect related diseases, which can lead to insufficient training. Therefore, we propose a new method (called MDMD) based on the aggregation of the metapath to efficiently and accurately predict potential drug-related microbes within the microbes-disease-drug network.

Circular RNAs (circRNAs) play a significant role in some diseases by acting as transcription templates. Therefore, analyzing the interaction mechanism between circRNA and RNA-binding proteins (RBPs) has far-reaching implications for the prevention and treatment of diseases. Existing models for circRNA-RBP identification most adopt CNN, RNN, or their variants as feature extractors. Most of them have drawbacks such as poor parallelism, insufficient stability, and inability to capture long-term dependence. To address these issues, we designed a Seq_transformer module to extract deep semantic features and then propose a CircRNA-RBP identification model based on Sequence Self-attention with Pre-normalization. We test it on 37 circRNA datasets and 31 linear RNA datasets using the same set of hyperparameters, and the overall performance of the proposed model is highly competitive and, in some cases, significantly out-performs state-of-the-art methods. The experimental results indicate that the proposed model is scalable, transformable, and can be applied to a wide range of applications without the need for task-oriented fine-tuning of parameters. The code is available at https://github.com/cc646201081/CircSSNN. Author summaryIn this paper, we propose a new method completely using the self-attention mechanism to capture deep semantic features of RNA sequences. On this basis, we construct a CircSSNN model for the cirRNA-RBP identification. The proposed model constructs a feature scheme by fusing circRNA sequence representations with statistical distributions, static local context, and dynamic global context. With a stable and efficient network architecture, the distance between any two positions in a sequence is reduced to a constant, so CircSSNN can quickly capture the long-term dependence and extract the deep semantic features. Experiments on 37 circRNA datasets show that the proposed model has overall advantages in stability, parallelism, and prediction performance. Keeping the network structure and hyperparameters unchanged, we directly apply CircSSNN to linRNA datasets. The favorable results show that CircSSNN can be transformed simply and efficiently without task-oriented tuning. In conclusion, CircSSNN can serve as an appealing circRNA-RBP identification tool with good identification performance, excellent scalability, and wide application scope, which is expected to reduce the professional threshold required for hyperparameter tuning in bioinformatics analysis.

Analysis of the sequential accumulation of genetic events, such as mutations and copy number alterations, is key to understanding disease dynamics and may provide insights into the design of targeted therapies. Oncogenetic graphical models are computational methods that use genetic event profiles from cross-sectional genomic data to infer the statistical dependencies between events and thereby deduce their temporal order of occurrence. Existing research focuses mainly on the development of graph structure learning algorithms. However, no algorithm explicitly links the oncogenetic graph with the temporal differences of samples in an analytic way. In this paper, we propose a novel statistical framework Timed Hazard Networks (TimedHN), that treat progression times as hidden variables and jointly infers oncogenetic graph and pseudo-time order of samples. We modeled the accumulation process as a continuous-time Markov chain and developed an efficient gradient computation algorithm for the optimization. Experiment results using synthetic data showed that our method outperforms the state-of-the-art in graph reconstruction. We highlighted the differences between TimedHN and competing methods on a luminal breast cancer dataset and illustrated the potential utility of the proposed method. Implementation and data are available at https://github.com/puar-playground/TimedHN

Language Models (LM) have been extensively utilized for learning DNA sequence patterns and generating synthetic sequences. In this paper, we present a novel approach for the generation of synthetic DNA data using pangenomes in combination with LM. We introduce three innovative pangenome-based tokenization schemes that enhance DNA sequence generation. Our experimental results demonstrate the superiority of pangenome-based tokenization over classical methods in generating high-utility synthetic DNA sequences, highlighting significant improvements in training efficiency and sequence quality.

Predicting potential links in heterogeneous biomedical networks (HBNs) can greatly benefit various important biomedical problem. However, the self-supervised representation learning for link prediction in HBNs has been slightly explored in previous researches. Therefore, this study proposes a two-level self-supervised representation learning, namely selfRL, for link prediction in heterogeneous biomedical networks. The meta path detection-based self-supervised learning task is proposed to learn representation vectors that can capture the global-level structure and semantic feature in HBNs. The vertex entity mask-based self-supervised learning mechanism is designed to enhance local association of vertices. Finally, the representations from two tasks are concatenated to generate high-quality representation vectors. The results of link prediction on six datasets show selfRL outperforms 25 state-of-the-art methods. In particular, selfRL reveals great performance with results close to 1 in terms of AUC and AUPR on the NeoDTI-net dataset. In addition, the PubMed publications demonstrate that nine out of ten drugs screened by selfRL can inhibit the cytokine storm in COVID-19 patients. In summary, selfRL provides a general frame-work that develops self-supervised learning tasks with unlabeled data to obtain promising representations for improving link prediction.

AO_SCPLOWBSTRACTC_SCPLOWRecent progress in machine learning provides competitive methods for bioinformatics in many traditional topics, such as transcriptomes sequence and single-cell analysis. However, discovering biomedical correlation of cells that are present across large-scale data sets remains challenging. Our attention-based neural network module with 300 million parameters is able to capture biological knowledge in a data-driven way. The module contains high-quality embedding, taxonomy analysis and similarity measurement. We tested the model on Mouse Brain Atlas, which consists of 160,000 cells and 25,000 genes. Our module obtained some interesting findings that have been verified by biologists and got better performance when benchmarked against autoencoder and principal components analysis.

The spread of cancer cells from the primary tumor to distant anatomical sites, known as tumor metastasis, poses a significant challenge in clinical prognosis, impairing treatment efficacy and reducing patient survival time. Current methods for predicting and diagnosing tumor metastasis rely heavily on extensive examinations and subjective clinical judgments. Accurate and rapid prediction of tumor metastasis likelihood remains an unresolved challenge, crucial for guiding effective clinical interventions and extending patient survival. Additionally, identifying key genes highly associated with metastasis probability is a pressing issue, essential for providing valuable insights into the potential identification of tumor metastasis-specific biomarkers. We developed MetaGXplore, a pioneering Graph Convolutional Neural Network (GCN)-based framework designed to predict metastasis probability by integrating pan-cancer multi-omic datasets with a protein-protein interaction network, while also identifying key genes involved in the metastatic process. Multi-omics datasets offer a comprehensive view of cancer biology, enhancing accuracy in metastasis forecasting through superior deep learning algorithms. Our classification model results were interpreted with GNNExplainer. The efficacy of MetaGXplore was validated via model evaluations, graph structure analysis, and multi-omics data assessment. Enrichment analysis of key genes further explored their biological roles.

Unsupervised learning of node representations from knowledge graphs is critical for numerous downstream tasks, ranging from large-scale graph analysis to measuring semantic similarity between nodes. This study presents gGN as a novel representation that defines graph nodes as Gaussian distributions. Unlike existing representations that approximate such distributions using diagonal covariance matrices, our proposal approximates them using low-rank perturbations. We demonstrate that this low-rank approximation is more expressive and better suited to represent complex asymmetric relations between nodes. In addition, we provide a computationally affordable algorithm for learning the low-rank representations in an unsupervised fashion. This learning algorithm uses a novel loss function based on the reverse Kullback-Leibler divergence and two ranking metrics whose joint minimization results in node representations that preserve not only node depths but also local and global asymmetric relationships between nodes. We assessed the representation power of the low-rank approximation with an in-depth systematic empirical study. The results show that our proposal was significantly better than the diagonal approximation for preserving graph structures. Moreover, gGN also outperformed 17 methods on the downstream task of measuring semantic similarity between graph nodes.

Tumor stratification, which aims at clustering tumors into biologically meaningful subtypes, is the key step towards personalized treatment. Large-scale profiled cancer genomics data enables us to develop computational methods for tumor stratification. However, most of the existing approaches only considered tumors from an individual cancer type during clustering, leading to the overlook of common patterns across cancer types and the vulnerability to the noise within that cancer type. To address these challenges, we proposed cancerAlign to map tumors of the target cancer type into latent spaces of other source cancer types. These tumors were then clustered in each latent space rather than the original space in order to exploit shared patterns across cancer types. Due to the lack of aligned tumor samples across cancer types, cancerAlign used adversarial learning to learn the mapping at the population level. It then used consensus clustering to integrate cluster labels from different source cancer types. We evaluated cancerAlign on 7,134 tumors spanning 24 cancer types from TCGA and observed substantial improvement on tumor stratification and cancer gene prioritization. We further revealed the transferability across cancer types, which reflected the similarity among them based on the somatic mutation profile. cancerAlign is an unsupervised approach that provides deeper insights into the heterogeneous and rapidly accumulating somatic mutation profile and can be also applied to other genome-scale molecular information. Availabilityhttps://github.com/bowen-gao/cancerAlign

Molecule property prediction based on computational strategy plays a key role in the process of drug discovery and design, such as DFT. Yet, these traditional methods are time-consuming and labour-intensive, which cant satisfy the need of biomedicine. Thanks to the development of deep learning, there are many variants of Graph Neural Networks (GNN) for molecule representation learning. However, whether the existed well-perform graph-based methods have a number of parameters, or the light models cant achieve good grades on various tasks. In order to manage the trade-off between efficiency and performance, we propose a novel model architecture, CoAtGIN, using both Convolution and Attention. On the local level, k-hop convolution is designed to capture long-range neighbour information. On the global level, besides using the virtual node to pass identical messages, we utilize linear attention to aggregate global graph representation according to the importance of each node and edge. In the recent OGB Large-Scale Benchmark, CoAtGIN achieves the 0.0933 Mean Absolute Error (MAE) on the large-scale dataset PCQM4Mv2 with only 5.6 M model parameters. Moreover, using the linear attention block improves the performance, which helps to capture the global representation.

Differentiating the intrinsic subtypes of breast cancer is crucial for deciding the best treatment strategy. Deep learning can predict the subtypes from genetic information more accurately than conventional statistical methods, but to date, deep learning has not been directly utilized to examine which genes are associated with which subtypes. To clarify the mechanisms embedded in the intrinsic subtypes, we developed an explainable deep learning model called a point-wise linear (PWL) model that generates a custom-made logistic regression for each patient. Logistic regression, which is familiar to both physicians and medical informatics researchers, allows us to analyze the importance of the feature variables, and the PWL model harnesses these practical abilities of logistic regression. In this study, we show that analyzing breast cancer subtypes is clinically beneficial for patients and one of the best ways to validate the capability of the PWL model. First, we trained the PWL model with RNA-seq data to predict PAM50 intrinsic subtypes and applied it to the 41/50 genes of PAM50 through the subtype prediction task. Second, we developed a deep enrichment analysis method to reveal the relationships between the PAM50 subtypes and the copy numbers of breast cancer. Our findings showed that the PWL model utilized genes relevant to the cell cycle-related pathways. These preliminary successes in breast cancer subtype analysis demonstrate the potential of our analysis strategy to clarify the mechanisms underlying breast cancer and improve overall clinical outcomes.

Binding sites are concave surfaces on proteins that bind to small molecules called ligands. Types of molecules that bind to the protein determine its biological function. Meanwhile, the binding process between small molecules and the protein is also crucial to various biological functionalities. Therefore, identifying and classifying such binding sites would enormously contribute to biomedical applications such as drug repurposing. Deep learning is a modern artificial intelligence technology. It utilizes deep neural networks to handle complex tasks such as image classification and language translation. Previous work has proven the capability of deep learning models handle binding sites wherein the binding sites are represented as pixels or voxels. Graph neural networks (GNNs) are deep learning models that operate on graphs. GNNs are promising for handling binding sites related tasks - provided there is an adequate graph representation to model the binding sties. In this communication, we describe a GNN-based computational method, GraphSite, that utilizes a novel graph representation of ligand-binding sites. A state-of-the-art GNN model is trained to capture the intrinsic characteristics of these binding sites and classify them. Our model generalizes well to unseen data and achieves test accuracy of 81.28% on classifying 14 binding site classes.